CLI improvements by dtrawins · Pull Request #4368 · openvinotoolkit/model_server

dtrawins · 2026-07-12T00:53:23Z

🛠 Summary

Allow updating configuration for local generative models in simplified way
Allow adding to config.json classic models including their extra parameters
Corrections in --help output

🧪 Checklist

Unit tests added.
The documentation updated.
Change follows security best practices.
``

Copilot

Pull request overview

This PR improves OVMS CLI flows around local generative models by introducing a dedicated --configure mode for generating/updating graph.pbtxt, and by expanding --add_to_config to support adding classic model parameters into config.json. It also refreshes --help output and updates tests/docs accordingly.

Changes:

Added --configure mode (no server start) to create/update graph.pbtxt for a local model based on --task options.
Extended config-management (--add_to_config) to accept and persist additional model parameters, switching JSON emission to RapidJSON.
Updated task-specific CLI help/validation and added/updated unit tests and documentation.

Reviewed changes

Copilot reviewed 16 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/test/ovmsconfig_test.cpp	Removes obsolete death test that assumed extra params are forbidden with `--add_to_config`.
src/test/config_export_test.cpp	Updates expected exported JSON formatting.
src/test/config_export_full_test.cpp	Adds end-to-end tests for `--configure` and `--add_to_config --batch_size`.
src/server.cpp	Adds CONFIGURE_MODE runtime path to generate `graph.pbtxt` and exit.
src/graph_export/t2s_graph_cli_parser.cpp	Updates help text; adds `--vocoder`; allows CONFIGURE_MODE in parser flow.
src/graph_export/s2t_graph_cli_parser.cpp	Updates help text; allows CONFIGURE_MODE in parser flow.
src/graph_export/rerank_graph_cli_parser.cpp	Updates help text; allows CONFIGURE_MODE in parser flow.
src/graph_export/image_generation_graph_cli_parser.cpp	Updates help text; allows CONFIGURE_MODE in parser flow.
src/graph_export/graph_cli_parser.cpp	Updates help text; allows CONFIGURE_MODE in parser flow.
src/graph_export/embeddings_graph_cli_parser.cpp	Updates help text; allows CONFIGURE_MODE in parser flow.
src/config.hpp	Updates config-management argument validation signature to include export type.
src/config.cpp	Expands allowed CLI args for config add/remove and updates validation for CONFIGURE_MODE.
src/config_export_module/config_export.cpp	Emits config JSON via RapidJSON and supports optional model fields on export.
src/cli_parser.hpp	Declares configure-mode detection helper.
src/cli_parser.cpp	Adds `--configure`, adjusts help printing, and relaxes preprocessing/layout checks for add-to-config.
src/capi_frontend/server_settings.hpp	Adds `CONFIGURE_MODE` enum value.
docs/parameters.md	Documents configure mode and expanded `--add_to_config` parameter support.

+        if (exportType == ENABLE_MODEL) {
+            std::cerr << "Adding models to the configuration file does not support parameters: " << arguments << std::endl;
+        } else {
+            std::cerr << "Removing models from the configuration file allows passing only model_name parameter. Invalid parameters passed: " << arguments << std::endl;
+        }


        SPDLOG_DEBUG("model_repository_path: {}", config.getServerSettings().hfSettings.downloadPath);
        return;
    }
+    if (config.getServerSettings().serverMode == CONFIGURE_MODE) {
+        SPDLOG_DEBUG("model_path: {}", config.modelPath());
+        return;
+    }


+static void addJsonOrStringMember(rapidjson::Value& obj, const char* key, const std::string& value, rapidjson::Document::AllocatorType& alloc) {
+    rapidjson::Document parsed(&alloc);
+    if (!parsed.Parse(value.c_str()).HasParseError() && parsed.IsObject()) {
+        rapidjson::Value jsonValue(parsed, alloc);
+        obj.AddMember(rapidjson::Value(key, alloc), jsonValue, alloc);
+    } else {
+        obj.AddMember(rapidjson::Value(key, alloc), rapidjson::Value(value.c_str(), alloc), alloc);
+    }
+}
+
+static void addOptionalModelFields(rapidjson::Value& configObj, const ModelsSettingsImpl& modelSettings, rapidjson::Document::AllocatorType& alloc) {
+    if (!modelSettings.batchSize.empty())
+        configObj.AddMember("batch_size", rapidjson::Value(modelSettings.batchSize.c_str(), alloc), alloc);
+    if (!modelSettings.shape.empty())
+        addJsonOrStringMember(configObj, "shape", modelSettings.shape, alloc);
+    if (!modelSettings.layout.empty())
+        addJsonOrStringMember(configObj, "layout", modelSettings.layout, alloc);
+    if (modelSettings.mean.has_value())
+        configObj.AddMember("mean", rapidjson::Value(modelSettings.mean.value().c_str(), alloc), alloc);
+    if (modelSettings.scale.has_value())
+        configObj.AddMember("scale", rapidjson::Value(modelSettings.scale.value().c_str(), alloc), alloc);
+    if (modelSettings.colorFormat.has_value())
+        configObj.AddMember("color_format", rapidjson::Value(modelSettings.colorFormat.value().c_str(), alloc), alloc);
+    if (modelSettings.precision.has_value())
+        configObj.AddMember("precision", rapidjson::Value(modelSettings.precision.value().c_str(), alloc), alloc);
+    if (!modelSettings.modelVersionPolicy.empty())
+        addJsonOrStringMember(configObj, "model_version_policy", modelSettings.modelVersionPolicy, alloc);
+    if (modelSettings.nireq != 0)
+        configObj.AddMember("nireq", modelSettings.nireq, alloc);
+    if (!modelSettings.targetDevice.empty())
+        configObj.AddMember("target_device", rapidjson::Value(modelSettings.targetDevice.c_str(), alloc), alloc);
+    if (!modelSettings.pluginConfig.empty())
+        addJsonOrStringMember(configObj, "plugin_config", modelSettings.pluginConfig, alloc);
+}


mzegla · 2026-07-17T14:42:57Z

+static void addJsonOrStringMember(rapidjson::Value& obj, const char* key, const std::string& value, rapidjson::Document::AllocatorType& alloc) {
+    rapidjson::Document parsed(&alloc);
+    if (!parsed.Parse(value.c_str()).HasParseError() && parsed.IsObject()) {
+        rapidjson::Value jsonValue(parsed, alloc);
+        obj.AddMember(rapidjson::Value(key, alloc), jsonValue, alloc);
+    } else {
+        obj.AddMember(rapidjson::Value(key, alloc), rapidjson::Value(value.c_str(), alloc), alloc);
+    }
+}


That's a general rapidjson utility. Should be elsewhere. Somewhere in the common sections for JSON classes.

mzegla · 2026-07-17T14:47:32Z

-            ("vocoder",
-                "The vocoder model to use for text2speech. For example microsoft/speecht5_hifigan",
-                cxxopts::value<std::string>(),
-                "VOCODER");


Why that's removed? It is added in graph export files

moved to text2speech specific section

mzegla · 2026-07-17T14:50:10Z

    }
-    // Ovms Pull models mode || pull and start models mode
+    // Ovms Pull models mode || pull and start models mode || configure mode
    if (isHFPullOrPullAndStart(this->result) || isInMemoryGraphMode(this->result)) {


Isn't || isConfigureMode(this->result needed here to enter this condition? Is configure mode detached from (isHFPullOrPullAndStart(this->result) || isInMemoryGraphMode(this->result))?

mzegla · 2026-07-17T14:51:53Z

+        if (!status.ok()) {
+            SPDLOG_ERROR("Failed to create graph config: {}", status.string());
+            return status;
+        }
+        std::cout << "Graph: graph.pbtxt created in: " << modelPath << std::endl;


Mixing std cout and spdlog. If logger is available at this point, use it for all printing.

Copilot

Pull request overview

Copilot reviewed 20 out of 21 changed files in this pull request and generated 4 comments.

+        HFSettingsImpl hfSettings = config.getServerSettings().hfSettings;
+        hfSettings.exportSettings.modelPath = ".";



+## Configure mode options
+
+Configure mode creates or updates `graph.pbtxt` for a local model without starting the server. It requires `--model_path` and `--task` parameters along with task-specific options.


+| `--configure`           | `NA`         | Runs in configure mode to create or update `graph.pbtxt` for a local model. Does not start the server.                                              |
+| `--model_path`          | `string`     | Path to the local model directory where `graph.pbtxt` will be created.                                                                              |
+| `--model_name`          | `string`     | Optional. Name of the model as exposed by the server.                                                                                               |
+| `--task`                | `string`     | Task type for the model (`text_generation`, `embeddings`, `rerank`, `image_generation`, `text2speech`, `speech2text`).                               |


+        if (result->count("configure") && result->count("pull")) {
+            ss << "error parsing options - --configure cannot be used with --pull" << std::endl;
+            return std::make_pair(OVMS_EX_USAGE, ss.str());
+        }


dtrawins added 3 commits July 12, 2026 02:47

initial version

df1c0cd

docs

17e3825

style

2910c33

dtrawins added this to the 2026.3 milestone Jul 16, 2026

dtrawins added 3 commits July 16, 2026 15:08

Merge remote-tracking branch 'origin/main' into configure

0a6b255

fix tests

a21302a

more tests

b589fed

dtrawins marked this pull request as ready for review July 16, 2026 14:55

dtrawins requested review from Copilot, michalkulakowski, mzegla and rasapala July 16, 2026 14:55

Copilot started reviewing on behalf of dtrawins July 16, 2026 14:55 View session

Copilot AI reviewed Jul 16, 2026

View reviewed changes

dtrawins added 5 commits July 17, 2026 00:27

fix documentation

e6162b4

fix relative path in staring ovms with model_path

d86d18d

style

3f42739

Merge remote-tracking branch 'origin/main' into configure

162fc8f

more tests

a58ae06

michalkulakowski approved these changes Jul 17, 2026

View reviewed changes

mzegla reviewed Jul 17, 2026

View reviewed changes

dtrawins added 2 commits July 17, 2026 17:22

review changes

e879d17

build fix

a7cd8d2

mzegla approved these changes Jul 17, 2026

View reviewed changes

dtrawins added 3 commits July 17, 2026 21:29

adjust to default task

912910e

style

ebc919d

build fix

e358ba2

dtrawins added 2 commits July 17, 2026 23:47

style

d18e22d

tests

a5366d5

dtrawins requested a review from Copilot July 17, 2026 23:57

Copilot started reviewing on behalf of dtrawins July 17, 2026 23:57 View session

Copilot AI reviewed Jul 18, 2026

View reviewed changes

dtrawins merged commit cb34be4 into main Jul 18, 2026
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

CLI improvements#4368

CLI improvements#4368
dtrawins merged 18 commits into
mainfrom
configure

dtrawins commented Jul 12, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

mzegla Jul 17, 2026

Uh oh!

mzegla Jul 17, 2026

Uh oh!

dtrawins Jul 17, 2026

Uh oh!

mzegla Jul 17, 2026

Uh oh!

mzegla Jul 17, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		HFSettingsImpl hfSettings = config.getServerSettings().hfSettings;
		hfSettings.exportSettings.modelPath = ".";


		## Configure mode options

		Configure mode creates or updates `graph.pbtxt` for a local model without starting the server. It requires `--model_path` and `--task` parameters along with task-specific options.

Uh oh!

Conversation

dtrawins commented Jul 12, 2026

🛠 Summary

🧪 Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

mzegla Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

mzegla Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

dtrawins Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

mzegla Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

mzegla Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants